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AIRBORNE  PERFORMANCE  MEASUREMENT  ASSESSMENT: 
LOW  ALTITUDE  TACTICAL  FORMATION 
IN  TWO  OPERATING  ENVIRONMENTS 


1.  STUDY  1:  ROUTINE  OPERATING  ENVIRONMENT 

This  report  describes  the  development  of  procedures  for  measuring  pilot  performance  in  the  low 
altitude  tactical  formation  (LATF)  task.  The  LATF  performance  measurement  procedures  were  developed 
as  a  part  of  the  Air  Force  Skills  Maintenance  and  Reaquisition  Training  research  program  (Project  SMART). 
The  objective  of  Project  SMART  is  to  identify  and  define  pilot  skills  necessary  for  combat  readiness,  to 
develop  procedures  for  measuring  performance  on  these  skills,  and  to  determine  training  requirements  for 
maintenance  of  necessary  proficiency  levels.  The  development  of  performance  measurement  procedures  for 
LATF  differed  substantially  from  that  of  an  earlier  effort  by  Project  SMART  which  generated  performance 
measurement  procedures  for  the  pop-up  weapons  delivery  (see  Pierce,  DeMaio,  Eddowes,  &  Yates,  1979). 
The  pop-up  is  a  short  duration  (less  than  40  seconds)  maneuver,  which  can  be  described  by  a  sequence  of 
discrete  stages.  In  the  pop-up,  specified  parameters  define  performance  requirements  for  each  stage,  and  an 
objective  criterion,  bomb  score,  exists  against  which  performance  on  the  maneuver  can  be  evaluated.  These 
characteristics  make  the  pop-up  amenable  to  a  sequential  performance  analysis  of  the  type  used  by  Pierce  et 
al. 

On  the  other  hand  LATF  is  a  flexible,  long  duration  (over  one  hour)  task.  During  LATF,  subtasks  are 
performed  repeatedly  as  needed  and  in  no  particular  sequence.  Task  requirements  vary  from  sortie  to  sortie, 
and  they  also  vary  with  position  in  the  formation.  Because  of  this  flexibility,  mission  performance  is  not 
evaluated  as  easily  for  LATF  as  it  is  for  the  pop-up.  The  LATF  task  cannot  be  described  by  a  sequence  of 
stages  nor  defined  by  specified  parameters  as  can  the  pop-up.  Neither  can  LATF  performance  be  measured 
against  an  objective  criterion,  such  as  the  bomb  score  measure  used  with  the  pop-up. 

The  basic  objective  of  LATF  is  to  provide  ingress  to  a  high  threat  target  area  without  exposing  the 
formation  to  ground  based  or  airborne  threats.  This  objective  is  accomplished  through  the  use  of  two 
tactics:  (a)  flying  at  low  altitude  and  using  terrain  features  to  mask  the  formation  and  (b)  performing  in  a 
mutual  support  role  which  will  permit  members  of  the  formation  to  detect  and  neutralize  any  threat  which 
might  be  encountered.  In  the  routine,  operational  environment,  these  formation  tactics  are  not  amenable  to 
standardization  and  objective  measurement  due  to  the  lack  of  appropriate  measurement  hardware. 
Therefore,  it  was  decided  not  to  attempt  to  divide  the  LATF  mission  into  a  sequence  of  segments  which 
demand  proficiency  at  certain  skills.  Rather,  it  was  decided  to  investigate  the  contribution  of  various 
skills/proficiency  factors  to  overall  mission  quality,  without  considering  where  in  the  mission  sequence 
particular  skills  might  be  important. 

Development  of  the  evaluation  methodology  began  with  pilot  interviews  to  determine  skill  areas 
which  might  be  critical  to  mission  success.  An  evaluation  form  was  then  constructed  on  which  pilots  could 
rate  their  performance  on  the  skill  areas  and  on  the  mission  as  a  whole.  Rating  data  were  collected  on 
routine  LATF  missions  and  in  LATF  missions  at  the  simulated  combat  exercise,  Red  Flag. 

Objective 

The  present  study  was  conducted  to  determine  what  areas  of  skill  proficiency  are  required  for 
successful  LATF  flying.  It  also  investigated  how  well  routine  flying  prepared  pilots  for  flying  in  a  simulated 
combat  environment. 


II.  METHOD 


Subjects 


The  subjects  for  this  study  consisted  of  28  pilots  of  the  354  Tactical  Fighter  Squadron  (TFS)  on 
active  flying  status  at  Davis-Monthan  AFB.  All  subjects  but  five  were  rated  mission  ready  in  the  A-7 
aircraft.  Five  subjects  were  rated  minimum  qualified. 


Materials 


Research  materials  consisted  of  the  LATF  Individual  Performance  Evaluation  Form  shown  in 
Appendix  A.  This  form  was  constructed  at  the  start  of  the  research  effort  using  information  obtained  in 
interviews  with  members  of  the  354  TFS. 

Procedure 

At  the  start  of  the  research  effort,  six  pilots  from  the  354  TFS  were  interviewed  to  provide  a  data 
base  for  use  in  constructing  the  LATF  Individual  Performance  Evaluation  Form.  Selection  of  pilots  for 
interviews  was  based  on  availability.  The  range  of  flying  experience  of  these  pilots  was  from  fewer  than  100 
flying  hours  in  an  operational  A-7  unit  to  over  3000  flying  hours  in  the  A-7,  including  combat  experience. 

The  pilots  were  interviewed  informally  in  groups  of  two.  At  the  start  of  the  interviews,  the  pilots 
were  asked  to  describe  a  typical  LATF  mission,  using  a  map  and  other  visual  aids  to  describe  particular 
formation  procedures  (e.g.,  navigation  turns,  ridge  crossings)  and  points  in  the  mission  when  particular 
procedures  might  be  employed.  Next,  they  were  asked  what  pilot  skills  or  functions  they  felt  had  an  impact 
on  the  success  of  the  mission  or  on  the  quality  of  an  individual’s  performance  as  a  member  of  the 
formation. 

Three  general  skill  areas/pilot  functions  were  identified  as  being  important  to  LATF:  (a)  factors 
relating  to  the  mutual  support  role  and  to  the  defensibility  of  the  formation,  (b)  low  altitude  flying/terrain 
masking,  and  (c)  navigation.  Riot  functions  which  were  felt  to  describe  a  pilot’s  performance  in  the  mutual 
support  role  were  (a)  visual  lookout,  which  permits  the  formation  to  detect  a  threat,  and  (b)  maintaining 
position  in  the  formation,  which  permits  it  to  neutralize  the  threat.  Riots  did  not  feel  that  the  lowest 
altitude  actually  flown  during  the  mission  was  an  adequate  measure  of  low  altitude  flying  performance 
because  this  was  too  greatly  influenced  by  minimum  altitude  restrictions.  Therefore,  it  was  decided  to  use 
comfort  level  (CL)  and  minimum  altitude  capability  (MAC)  estimates  of  the  pilot’s  own  capacity  to  fly  at 
low  altitude  rather  than  to  use  the  lowest  altitude  actually  flown.  CL  is  defined  as  the  lowest  altitude  that 
members  of  the  formation  can  detect  a  threat  and  perform  intense,  evasive  maneuvering.  Minimum  altitude 
capability  is  defined  as  the  altitude  at  which  intense  maneuvering  is  severely  restricted  and  maintenance  of 
that  altitude  is  the  primary  defensive  response  ( Low  Altitude  Flying,  1978).  Factors  which  were  felt  to 
index  the  quality  of  navigation  performance  were  accuracy  of  the  formation’s  time  on  target  (TOT)  and 
distance  off  target  (DOT).  Some  pilots  expressed  doubt  that  either  of  these  factors  would  be  of  much 
importance  in  determining  overall  quality  of  routine  missions  because  only  the  flight  leader  was  responsible 
for  navigation  and  because  squadron  members  were  generally  very  familiar  with  all  routes  routinely  flown. 
However,  others  felt  that  all  formation  members  were  responsible  for  navigation  awareness  if  not  actual 
navigation. 

It  was  suggested  that  an  individual’s  performance  was  affected  by  that  of  the  other  element  member. 
If  one  element  member  maintained  position  well,  then  performance  of  the  other  member  would  be 
enhanced.  It  was  also  suggested  that  the  quality  of  the  formation  was  enhanced  when  members  were 
accustomed  to  flying  together.  Familiarity  with  element  member  was  not  felt  to  enhance  performance  on  a 
particular  task  but  was  felt  to  improve  the  coordination  and  integrity  of  the  formation. 

An  LATF  individual  performance  rating  form  was  constructed  which  included  all  of  the  factors 
determined  to  be  relevant  from  the  interview  data.  In  addition,  total  flying  hours,  days  since  last  LATF 
mission,  and  a  rating  of  overall  LATF  performance  on  the  mission  were  included.  Ratings  of  performance 
on  the  formation  tasks  were  made  on  an  8-point  scale  with  a  rating  of  1  being  least  positive  and  a  rating  of 
8  being  most  positive.  Four  measures  of  low  altitude  flying  performance  were  obtained:  MAC,  CL  (straight 
and  level),  CL  (in  turns),  and  CL  (over  rough  terrain):  these  were  given  in  feet  above  ground  level  (AGL). 
Navigation  measures,  TOT  and  DOT  were  in  seconds  of  error  and  1  /8-mile  of  error,  respectively. 

Following  each  LATF  mission,  a  researcher  was  on  hand  to  pass  out  and  collect  forms  and  to  answer 
any  questions  about  the  forms.  All  flight  members  rated  their  own  performance  on  all  variables  but  one. 


Each  individual  rated  the  performance  of  their  element  member  on  maintaining  position;  this  rating  was 
made  on  the  same  form  as  the  individual’s  self-ratings. 


III.  RESULTS 

From  July  to  September  1978,  106  evaluation  forms  were  collected  on  three-ship  and  four-ship 
formation  flights.  Of  these,  31  were  omitted  from  the  data  analysis  because  of  missing  data. 

Data  analysis  was  accomplished  by  performing  a  step-wise,  linear  multiple  regression  with  the  overall 
formation  performance  variable  as  the  criterion  variable  (Helmstadter,  1972).  This  analysis  provided  the 
best  fitting  linear  multiple  regression  equation  and  multiple  correlation  coefficient,  a  Pearson 
product-moment  correlation  matrix  for  all  variables,  and  the  means  and  variances  for  all  variables.  The 
multiple  correlation  coefficient  between  the  performance  rating  on  the  subtasks  and  the  overall  rating  was 
used  to  assess  the  reliability  of  the  measurement  system.  There  were  two  reasons  for  employing  this 
within-subject  measure  rather  than  the  correlation  between  element  members’  ratings.  The  first  reason  was 
that  several  variables  (e.g.,  visual  lookout,  comfort  level)  could  be  rated  only  by  the  individual  and  not  by 
the  other  element  member.  The  second  reason  was  that  flight  members  discussed  the  mission  both  during 
the  flight  and  during  the  debrief,  which  usually  preceded  filling  out  the  LATF  form.  It  was  felt  that  this 
discussion  might  cause  an  undue  inflation  of  the  between-member  correlation. 

The  results  of  the  regression  analysis  are  shown  in  Table  1.  A  highly  consistent  relationship  was 
obtained  between  pilots’  overall  performance  rating  and  their  ratings  on  formation  subtasks  (R=.92).  Four 
variables  were  found  to  contribute  to  the  pilots’  ratings  of  their  own  overall  formation  performance.  In 
order  of  the  magnitude  of  their  contribution  to  the  regression,  these  were  (a)  maintaining  position  (own 
performance),  (b)  visual  lookout,  (c)  maintaining  position  (element  member’s  performance),  and  (d) 
familiarity  with  element  member.  The  first  two  variables  showed  a  relatively  strong  relationship  to  the 
(overall  rating)  criterion.  Nearly  35%  of  the  contribution  of  the  visual  lookout  variable  to  the  regression  was 
due  to  correlation  with  the  criterion;  nearly  56%  of  the  contribution  of  the  maintaining  position  (self) 
variable  was  due  to  correlation  with  the  criterion.  By  contrast,  roughly  80%  of  the  contribution  of  the 
familiarity  and  maintaining  position  (element  member)  variables  was  due  to  correlation  with  other  variables 
in  the  regression  equation. 
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Table  I.  Multiple  Regression  Analysis  of  the  Formation  Performance 
Self-Rating  Data  for  Routine  Operations 


Correlation  Matrix 


Variable  Name 

EF 

VL 

CL 

PS 

PE 

OR 

EF 

1.0 

VL 

.46 

1.0 

CL 

-.17 

.01 

1.0 

PS 

.55 

.70 

.04 

1.0 

PE 

.49 

.66 

.23 

.77 

1.0 

OR 

.59 

.78 

.09 

.88 

.79 

1.0 

Basic  Regression  Statistics 

Standard  Error  of  Estimate  .53 

Multiple  R  .92 

Coefficient  of  Determination  .85 

Corrected  Coefficient  of  Determination  .84 


Variable 

Typa 

Regression 

Coefficient 

Std  Err  of 

Regression 

Coefficient 

Standardlzad 

Regrasalon 

Coafficlant 

Partial 

Correlation 

Coafficiant 

Partial  F  Value 
With  1  and  70 
Deg  Freedom 

Sig 

Level 

OR 

Dependent 

_ 

_ 

_ 

_ 

_ 

_ 

Constant 

.04 

.29 

.02 

.02 

.89 

EF 

Freevar 

.06 

.03 

.11 

.22 

3.77 

.06 

VL 

Freevar 

.29 

.07 

.27 

.43 

16.28 

.00 

CL 

Freevar 

.15 

.21 

PS 

Freevar 

.47 

.07 

.49 

.58 

36.08 

.00 

PE 

Freevar 

.18 

.07 

.18 

.27 

5.69 

.02 

Legend 

OR  Overall  Rating 

EF  Familiarity  with  Element  Member 
VL  Visual  Lookout 

CL  Comfort  Level 

PS  Maintaining  Position  (Self) 

PE  Maintaining  Position  (Element  Member) 

Note:  Overall  performance  rating  is  the  criterion  variable. 


IV.  DISCUSSION 

Two  subtasks  performed  during  the  LATF  mission  were  found  to  affect  a  pilot’s  ratings  of  overall 
mission  performance.  Both  of  these  tasks  are  related  to  the  mutual  support  role.  Visual  lookout  is  critical  to 
the  success  of  the  formation  because  it  is  this  function  which  permits  the  formation  to  detect  and  respond 
to  a  threat  before  it  is  too  late  to  respond  effectively  to  the  threat.  Maintaining  good  formation  position 
facilitates  the  mutual  support  function  in  two  ways.  The  first  is  that  it  facilitates  the  visual  lookout 
function.  The  formation  can  best  detect  a  threat  in  the  5  o’clock  to  7  o'clock  cone  if  the  rear  element 
members  are  line  abreast  and  at  5,000  to  50,000  feet  separation  (sec  Figure  1).  In  this  position,  each  pilot 
can  clear  the  element  member’s  6  o'clock  position  to  a  distance  of  over  8,000  feet.  As  the  separation 
between  the  two  aircraft  decreases,  the  distance  to  which  a  threat  can  approach  undetected  decreases.  If 


one  member  of  the  element  moves  ahead  of  line  abreast,  this  exposes  the  element  member’s  6  o’clock 
position  to  attack.  If  a  member  moves  behind  line  abreast,  then  that  member’s  own  6  o’clock  position  is 
exposed. 


C.  2  TOO  FAR  FORWARD  D.  2  TOO  FAR  BACK 


Note:  Figure  I A  is  correct  line  abreast  position  for  a  two-ship  formation  element.  Separation  between  element  members  is 
5,000  to  10,000  feet.  Figure  IB,  C,and  D  represent  incorrect  position  and  the  blind  area  in  the  5  to  7  o'clock  cone  moves 
closer  to  one  or  both  aircraft.  Also  improper  position  impairs  ability  of  the  element  to  respond  to  a  threat. 

Figure  I.  Effect  of  position  on  formation  performance. 


Good  formation  position  is  also  necessary  for  proper  response  to  a  threat,  once  it  has  been  detected. 
If  two  members  of  an  element  have  inadequate  separation,  they  can  be  attacked  simultaneously  and  will 
impede  each  other’s  response.  If  they  have  too  much  separation,  too  much  time  will  be  required  for  one 
member  to  get  into  position  to  respond  to  an  attack  on  the  other. 

The  maintaining  position  (element  member)  variable  was  also  found  to  be  a  significant  contributor  to 
the  overall  performance  rating.  That  is,  the  pilots  rated  their  own  overall  performance  higher  when  they 
rated  their  element  member’s  performance  high.  This  variable  does  not  have  its  effect  through  direct  action 
on  a  role  of  the  formation  but  rather  through  its  effect  on  pilot  workload.  When  the  element  member 
maintains  good  formation  position,  this  reduces  the  amount  of  effort  that  the  pilot  must  expend  to 
maintain  position.  This  workload  reduction  permits  the  pilot  to  devote  more  effort  to  other  formation 
tasks.  This  effect  can  be  seen  in  the  high  correlation  between  position  (element  member)  and  the  pilot 
performance  variables. 

The  last  contributor  to  the  regression  equation,  familiarity  with  element  member,  differs  from  the 
others  in  that  it  is  not  a  performance  or  a  skill  variable.  It  also  differs  from  the  individual  performance 
variables  in  that  a  substantial  part  of  its  contribution  (80%)  comes  from  its  correlation  with  other  variables 
in  the  regression  equation.  This  variable  also  has  its  effect  through  reduction  of  pilot  workload.  When 
members  of  the  formation  are  familiar  with  each  other,  they  are  better  able  to  anticipate  each  other’s 
actions  and  to  plan  for  them.  They  are  also  less  likely  to  misunderstand  each  other’s  signals  or  to  nake 
inappropriate  responses. 

The  navigation  variables  both  showed  a  very  high  degree  of  accuracy  but  neither  contributed  to  the 
regression  equation.  Eighty-three  percent  reported  a  distance  error  of  1  /8  mile  or  less;  60%  reported  a  time 
on  target  error  less  than  20  seconds.  A  separate  analysis  was  not  conducted  on  flight  leaders’  data  since  the 
high  degree  of  navigation  accuracy  and  low  variability  obtained  made  it  seem  unlikely  that  these  variables 
would  make  a  significant  contribution  even  for  this  group.  The  low  altitude  flying  ability  variables,  too,  did 
not  contribute  significantly  to  the  regression  equation.  From  these  data,  it  is  not  possible  to  determine 
whether  this  was  due  to  minimum  altitude  restrictions  or  was  an  accurate  assessment  of  the  perceived 
importance  of  these  variables. 

V.  STUDY  II:  RED  FLAG  OPERATING  ENVIRONMENT 

During  the  course  of  the  data  collection  effort,  several  pilots  expressed  the  opinion  that  more 
enlightening  data  on  LATF  could  be  obtained  at  Red  Flag.  Red  Flag  is  an  operation  designed  to  simulate 
the  high  threat  combat  environment.  Pilots  plan  and  fly  combat  sorties  over  a  2-week  period.  On  each 
mission,  they  are  subject  to  attack  by  groundbased  threats  and  airborne  “aggressor”  forces.  This  procedure 
provides  the  squadron  an  opportunity  to  test  its  ability  to  function  effectively  in  an  environment  of  high 
threat  (Joss,  1978).  It  also  puts  great  pressure  on  individual  squadron  members  to  perform  at  their  best. 

At  the  invitation  of  the  354  TFS,  Project  SMART  researchers  continued  data  collection  with  the 
squadron  during  its  Red  Flag  exercise.  The  purpose  of  the  study  was  to  evaluate  LATF  performance  under 
conditions  simulating  those  of  actual  combat. 

VI.  METHOD 

Subjects 

The  subjects  were  20  pilots  of  the  354  TFS  who  were  detailed  to  Red  Flag.  They  were  some  of  the 
same  pilots  who  participated  in  the  Davis-Monthan  phase  of  the  research. 

Materials 

The  materials  consisted  of  a  combination  of  LATF  performance  evaluation  and  pop-up  weapons 
delivery  performance  evaluation  on  one  form.  During  normal  operations,  data  on  the  two  tasks  were 


collected  on  separate  forms;  however,  it  was  felt  that  collection  at  Red  Flag  would  be  expedited  by 
combining  both  tasks  on  one  form.  In  combining  the  forms  for  use  at  Red  Flag,  it  was  necessary  to 
eliminate  some  variables.  Since  analysis  of  the  Davis-Monthan  data  had  not  yet  been  completed,  this  was 
accomplished  by  eliminating  non-performance  variables  (e.g.,  time  since  last  low  altitude  mission)  and  those 
which  showed  a  low  individual  correlation  with  the  overall  rating  (e.g.,  MAC).  In  the  present  report,  only 
the  LATF  data  were  considered.  An  example  of  the  form  is  shown  in  Appendix  B. 

Procedure 

Squadron  personnel  attended  Red  Flag  in  two  groups.  Each  group  included  roughly  one-half  of  the 
pilots  in  the  squadron  available  for  this  assignment  and  stayed  at  Red  Flag  for  2  weeks.  Thus,  the  data 
collection  period  was  1  month. 

A  Project  SMART  researcher  was  on  hand  at  the  end  of  each  LATF  mission  to  pass  out  and  collect 
the  forms.  Since  all  of  the  pilots  had  participated  in  the  Davis-Monthan  phase  of  the  study,  they  were 
familiar  with  the  forms  and  procedure. 


VH.  RESULTS 

In  condensing  the  LATF  form  for  use  at  Red  Flag,  the  variable,  maintaining  position  (element 
member’s  performance),  was  omitted.  Therefore,  its  importance  to  the  overall  rating  could  not  be  assessed. 
As  with  the  data  from  the  Davis-Monthan  phase,  a  step-wise,  linear  multiple  regression  analysis  was 
performed  on  the  data  from  30  LATF  performance  evaluations.  The  overall  performance  rating  was  again 
used  at  the  criterion  variable. 

Mean  ratings  obtained  at  Red  Flag  were  compared  with  those  obtained  at  Davis-Monthan.  Familiarity 
with  element  member  was  rated  significantly  higher  at  Red  Flag  (see  Table  2),  and  CL  was  significantly 
lower  than  that  obtained  at  Davis-Monthan. 


Table  2.  Comparison  of  Mean  Self-Ratings  for  Routine 
Operations  and  Simulated  High-Threat  Conditions 


Davis-Monthan 

Red  Flag 

Variable 

Mean 

SO 

Mean 

SD 

ttt3) 

P 

Familiarity 

4.6 

2.3 

5.6 

1.7 

2.13 

.05 

Visual  Lookout 

5.0 

1.3 

5.4 

1.5 

1J4 

Comfort  Level  (Ft) 

21 1.0 

138.0 

155.0 

56.0 

2.12 

.05 

Position  (Self) 

Position  (Element 

5.5 

1.4 

5.7 

1.8 

0.6 

.5 

Member) 

4.2 

1.3 

- 

- 

Overall 

5.4 

1.3 

5.7 

1.5 

1.01 

.35 

The  best  fitting  linear  multiple  regression  equation  obtained  for  the  Red  Flag  data  differed  from  that 
obtained  for  the  Davis-Monthan  data  in  two  ways.  The  first  was  that  the  proportional  contribution  of  the 
visual  lookout  vartable  in  the  regression  equation  increased  substantially  at  Red  Flag  (34%  vs.  21%).  The 
second  was  that  the  variable,  familiarity  with  element  member,  dropped  out  of  the  regression  equation  (see 
Table  3).  As  mentioned  previously,  data  were  not  collected  on  the  variable,  maintaining  position  (element 
member). 
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Table  3.  Multiple  Analysis  of  Formation  Performance  Self-Rating 
Data  for  Simulated  High-Threat  Conditions 


Correlation  Matrix 


Variable  Name 

EF 

VL 

CL 

PS 

EF 

1.0 

VL 

.21 

1.0 

CL 

.31 

.25 

1.0 

PS 

.23 

.70 

.28 

1.0 

OR 

.25 

.79 

.31 

.81 

Basic  Regression  Statistics 

Standard  Error  of  Estimate  .77 

Multiple  R  .87 

Coefficient  of  Determination  .76 

Corrected  Coefficient  of  Determination  .74 


Variable 

Typ# 

Regression 

Coefficient 

Std  Err  of 
Regression 
Coefficient 

Standardized 

Regression 

Coefficient 

Partial 

Correlation 

Coefficient 

Partial  F  Value 
With  1  and  70 
Deg  Freedom 

Sig 

Level 

OR 

Dependent 

Constant 

.88 

.54 

30 

2.68 

•ii 

EF 

Freevar 

.08 

.69 

VL 

Freevar 

.43 

.13 

.43 

.53 

10.76 

.00 

CL 

Freevar 

.12 

.55 

PS 

Freevar 

.43 

.11 

.53 

.60 

15.40 

.00 

Legend 

OR  Overall  Rating 

EF  Familiarity  with  Element  Member 
VL  Visual  Lookout 

CL  Comfort  Level 

PS  Maintaining  Position  (Self) 

PE  Maintaining  Position  (Element  Member) 

Note:  Overall  performance  rating  is  the  criterion  variable. 


VIII.  DISCUSSION 

The  increase  in  the  familiarity  rating  is  a  product  of  the  difference  in  the  strategy  employed  by  flight 
leaders  in  making  formation  assignments  at  Red  Flag.  Under  normal  operating  conditions,  formation 
assignments  are  made  on  the  basis  of  training  and  scheduling  requirements.  At  Red  Flag,  formation 
assignments  are  made  to  optimize  the  effectiveness  of  the  squadron.  Thus,  a  particularly  strong  squadron 
member  may  be  assigned  to  a  weaker  member  whose  difficulties  the  stronger  understands  and  for  which 
that  individual  can  compensate,  or  two  members  who  arc  particularly  compatible  may  be  assigned  together. 
On  the  whole,  the  strategy  leads  to  formations  consisting  of  members  who  are  more  familiar  with  each 
other. 

Pilots  reported  a  drop  of  roughly  56  feet  in  CL  at  Red  Flag.  This  drop  occurred  despite  the  fact  that 
pilots  were  less  familiar  with  the  terrain.  This  would  be  expected  to  make  pilots  feel  less  comfortable  at  low 


altitude.  Yet,  they  report  feeling  comfortable  at  lower  altitude  than  during  routine  operations.  This  suggests 
that  the  lowering  of  CL  does  not  reflect  the  fact  that  pilots  are  more  comfortable  at  lower  altitude  but 
rather  that  they  are  less  comfortable  at  higher  altitude.  During  routine  operations,  flying  at  too  high  an 
altitude  is  not  the  best  practice  but  is  a  conservative  error.  At  Red  Rag,  such  an  error  is  more  serious  and 
can  compromise  mission  success  or  result  in  being  “shot  down”  by  anti  aircraft  artillery  or  by  an  airborne 
aggressor.  Thus,  it  is  much  more  important  to  be  “comfortable”  at  low  altitude  in  the  simulated  combat 
environment  of  Red  Flag. 

Despite  the  increased  importance  of  low  altitude  flying  as  reflected  by  the  lowering  of  CL,  CL  does 
not  contribute  significantly  to  the  regression  equation.  These  data  do  not  fully  answer  the  question  of  why 
low  altitude  flying  is  not  seen  as  an  important  contributor  to  LATF,  but  they  do  suggest  that  minimum 
altitude  restrictions  alone  are  not  the  reason.  At  Red  Rag,  there  are  fewer  altitude  restrictions,  although 
there  are  minimums,  and  restrictions  are  imposed  by  the  formation  itself.  Thus  at  Red  Rag,  CL,  which  is  a 
measure  of  individual  flying  ability,  decreases;  but  this  change  is  not  seen  as  important  to  a  pilot’s 
performance  as  part  of  the  formation. 

A  pilot’s  response  to  the  higher  threat  environment  at  Red  Rag  can  also  be  seen  in  the  importance 
given  to  the  visual  lookout  task.  The  proportional  contribution  of  the  visual  lookout  variable  in  the 
regression  equation  increased  from  21%  at  Davis-Monthan  to  34%  at  Red  Rag.  Unlike  the  routine  training 
environment,  the  Red  Rag  environment  contains  actual  threats  which  must  be  detected  and  neutralized  if  a 
mission  is  to  be  completed  successfully.  Thus,  there  is  a  real  pressure  to  perform  the  visual  lookout  task 
well.  In  addition,  the  presence  of  threats  provides  a  means  for  the  pilots  to  assess  more  precisely  the  quality 
of  their  own  performance  at  this  task. 

The  last  variable  to  be  considered  was  not  found  to  be  a  significant  contributor  to  pilot  rating  of 
performance  at  Red  Flag.  That  is  familiarity  with  element  member.  In  discussing  the  Davis-Monthan  data,  it 
was  suggested  that  the  familiarity  variable  has  its  effect  through  a  reduction  in  workload  when  flying  with  a 
familiar  element  member.  The  data  from  Red  Rag  suggest  that  flight  leaders  take  advantage  of  this 
workload  reduction  in  making  formation  assignments  for  optimal  effectiveness.  This  can  be  seen  in  the 
significantly  higher  familiarity  rating  obtained  at  Red  Rag.  However,  despite  this  strategy,  familiarity  was 
not  found  to  be  a  significant  contributor  to  the  quality  of  performance.  It  might  be  argued  that  the  failure 
of  familiarity  to  contribute  to  mission  quality  is  a  result  of  the  assignment  strategy.  However,  since  the 
standard  deviation  of  the  familiarity  variable  at  Red  Rag  did  not  differ  from  that  obtained  at 
Davis-Monthan  (F(l,104)=1.8,  p=-05)  where  familiarity  did  contribute  to  the  regression,  this  explanation 
can  be  ruled  out. 

The  failure  of  familiarity  to  contribute  overall  performance  quality  suggests  that  familiarity  with 
element  member  did  not  serve  to  reduce  pilot  workload  in  the  high  threat  environment.  This  might  be 
expected  if,  at  Red  Rag,  pilots  did  not  follow  the  behavior  patterns  which  they  follow  in  routine 
operations.  Being  familiar  with  one’s  element  member  wfll  not  provide  any  advantage  if  that  member  does 
not  behave  as  expected.  Thus,  the  failure  of  familiarity  to  contribute  to  the  regression  equation  suggests 
that  pilot  behavior  becomes  more  variable  to  some  degree  upon  entering  the  Red  Rag  environment.  This 
conclusion  is  supported  by  numerous  reports  of  atypical,  and  often  counterproductive,  behavior  at  Red 
Rag.  It  is  not  possible  to  determine  from  the  present  data  how  long  the  period  of  initial  reorientation  lasts 
at  Red  Rag.  However,  data  from  actual  combat  situations  suggest  that  reorientation  may  persist  for  the 
first  ten  sorties  (Joss,  1978).  If  the  period  of  reorientation  were  to  last  this  long  at  Red  Rag,  then  it  would 
encompass  nearly  the  entire  length  of  the  Red  Rag  training  exercise.  Such  a  situation  might  greatly  impair 
the  effectiveness  of  the  exercise  in  preparing  pilots  to  enter  the  combat  environment. 

There  are  two  possible  approaches  which  might  be  taken  to  minimize  the  need  for  pilot  reorientation 
and  so  to  maximize  the  training  effectiveness  of  exercises  like  Red  Rag.  One  would  be  to  increase  the 
length  of  the  exercise,  giving  pilots  additional  time  to  adapt  to  the  increased  stress  of  the  exercise.  The 
second  would  be  to  increase  the  realism  of  the  routine  training  done  before  the  exercise.  This  could  be 
accomplished  by  employing  routine  “Red  Baron”  missions  on  which  the  formation  is  subjected  to  a 


surprise  airborne  attack.  This  approach  is  currently  used  to  a  limited  extent.  However,  its  effectiveness  could 
be  enhanced  if  it  were  practiced  more  regularly.  In  addition,  frequently  overlooked  procedures  might  be 
practiced  to  improve  the  simulation  of  unfamiliar  types  of  combat-related  missions  (Millard  &  Michel, 
1978).  Routine  training  realism  might  also  be  enhanced  by  conducting  “mini-Red  Flag”  exercises  in  which 
aggressors  operate  with  the  squadron  at  its  home  base.  This  procedure,  too,  is  in  limited  use,  but  scheduling 
of  the  exercises  is  not  designed  to  integrate  them  optimally  with  Red  Rag  operations.  Thus,  there  can  be  a 
long  interval  between  the  two  operations  which  may  reduce  the  effectiveness  of  both. 

Any  approach  to  increasing  the  realism  of  combat  readiness  training  involves  risk  of  increased  costs, 
increased  accident  rates,  or  disruption  of  normal  operations.  Implementation  of  such  programs  therefore 
must  be  preceded  by  thorough  research  to  insure  that  training  gains  are  not  offset  by  these  negative  factors. 
The  present  research  suggests  that  greater  realism  could  improve  training  by  increasing  pilots’  ability  to 
adapt  quickly  to  the  combat  environment.  Further  research  is  required  to  determine  the  best  approach  to 
increasing  training  realism  in  order  to  achieve  optimal  training  effectiveness. 


IX.  CONCLUSIONS 

1.  Pilot  performance  self-rating  methodology  was  successfully  applied  to  the  low  altitude  tactical 
formation  task.  Pilot  performance  ratings  were  found  to  have  high  internal  consistency. 

2.  Skill  areas  found  to  be  important  to  the  quality  of  an  individual’s  formation  performance  were 
related  to  the  mutual  support  role  of  the  formation.  Specific  skill  areas  were  (a)  visual  lookout,  clearing  the 
area  for  threats  and  (b)  maintaining  position  in  the  formation. 

3.  The  quality  of  an  individual’s  performance  was  found  to  be  affected  by  the  other  member  of  the 
element.  An  individual’s  performance  rating  increased  when  that  individual’s  rating  of  the  element 
member’s  performance  increased.  Also,  individuals  rated  their  performance  better  when  they  flew  with  a 
familiar  element  member. 

4.  Data  suggested  that,  when  placed  in  a  simulated  combat  environment,  pilots  may  experience  a 
period  of  reorientation  and  atypical  behavior  similar  to  that  reportedly  experienced  in  the  actual  combat 
situation.  Further  research  is  planned  to  determine  to  what  degree  this  reorientation  period  impairs  training 
effectiveness  and  to  assess  the  utility  of  different  training  techniques  for  reducing  this  impairment. 
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APPENDIX  A:  LOW  LEVEL  TACTICAL  FORMATION  PERFORMANCE  QUESTIONNAIRE 


Pilot  ID _ Flight _ Date _ 

1 .  What  type  of  formation  was  flown  on  this  mission? 

_ Box  _ Wedge  _ Fluid  2  _ Other  (Specify) 

2.  What  was  your  position  in  this  flight?  _ Flight  Leader 

_ Element  Leader _ Wingman  Front _ Wingman  Rear  _ Fighting  Wing 

3.  Rate  your  familiarity  with  your  element  member? 

N/A  Unfamiliar  Very  Familiar 

0  _ 

1  2  3  4  5  6  7  8 

(Circle  one  numeral) 

4.  How  accurately  did  your  flight  arrive  at  its  destination: 

a.  Arrival  Time:  _ ±10  Sec  _ ±10-20  Sec  _ ±20— 40  Sec 

_ ±40-60  Sec  _ ±60-90  Sec  _ +90-1 20  Sec  _ Unsure 

b.  Distance:  _ 1/8  Mi  _ 1/8 -1/4  Mi  _ 1/4 -1/2  Mi 

_ 1/2-1  Mi  _ 1-1  1/2  Mi  _ >11/2  Mi  _ Unsure 

5.  How  many  times  was  your  flight  engaged/detected  during  the  mission? 
a.  Rate  your  performance  in  providing  visual  lookout  support . 

Unsatisfactory  Excellent 

1  2  3  4  5  6  7  8 

(Circle  one  numeral) 

6.  What  do  you  feel  was  your  Minimum  Altitude  Capability  on  this  mission  (lowest  altitude  flown 

with  no  concurrent  task)? _ _  Ft 

7.  What  do  you  feel  was  your  Comfort  Level  on  this  mission  (lowest  altitude  at  which  concurrent 

tasks  could  be  performed)?  For  straight  flight? - Ft 

For  turns? _ Ft  Over  rough  terrain? _ Ft 

8.  Formation  maneuvering: 

a.  Rate  your  performance  in  maintaining  position: 

N/A  Unsatisfactory  Excellent 

0  _ 

1  2  3  4  5  6  7  8 

(Circle  one  numeral) 

b.  Rate  your  element  members  performance  in  maintaining  position: 

N/A  Unsatisfactory  ExceUent 

1  2  3  4  5  6  7  8 

(Circle  one  numeral) 

9.  Rate  your  overall  performance  on  the  low  level  tactical  formation  phase  of  this  mission: 

N/A  Unsatisfactory  Excellent 

0  _ 

1  2  3  4  5  6  7  8 

(Circle  one  numeral) 

10.  Error  Analysis: _ 

_ 


APPENDIX  B:  PILOT  SELF-ASSESSMENT 


Mot  ID 


Flight 


Date 


1 


Pop-Up 

Pass 

2  3  4 

_  _  _  LOS 

_  _  _  Airspeed 

_  _  _  Altitude 

_  _  _  Bomb  Score 

Pilot  Ratings:  S  -  Satisfactory  M  -  Marginal  U  -  Unsatisfactory 

_  _  _  PUP  Acquisition 

_  _  _  Pull  Up  Point 

_  _  _  Target  Acquisition 

_  _  _  Airspeed 

_  _  _  Dive  Angle 

_  _  _  Track  Time 

_  _  _  Release  Parameters 

_  _  _  Recovery 

_  _  _  Overall  Performance 


Low  Level  Tactical  Formation 

Rate  the  following  (1  =  Poor;  8  = 

Excellent) 

1. 

Familiarity  with  element  member 

Unfamiliar 

(Circle  one  numeral) 

Familiar 

1  2 

3 

4  5  6 

7 

8 

2. 

Time  on  Target 

OFF  TOT 

ON  TOT 

1  2 

3 

4  5  6 

7 

8 

3. 

Comfort  Level  (Lowest  Altitude  Flown  Comfortably): 

High 

(Circle  one  numeral) 

Low 

1  2 

3 

4  5  6 

7 

8 

4. 

Give  Comfort  Level: 

CL _ 

FT  AGI 

5. 

Visual  Lookout  (Check  6  O’Clock) 

N/A  Unsatisfactory 

(Circle  one  numeral) 

Satisfactory 

0  1  2 

3 

4  5  6 

7 

8 

6. 

Maintaining  Position: 

Unsatisfactory 

(Circle  one  numeral) 

Satisfactory 

1  2 

3 

4  5  6 

7 

8 

7. 

Overall  Low  Level  Performance: 

Unsatisfactory 

(Circle  one  numeral) 

Satisfactory 

1  2 

3 

4  5  6 

7 

8 

*us 
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